Can we make SYCL_CTS_ENABLE_EXT_ONEAPI_TESTS to set a default value for the tests, which can be overridden by test specific macro?

Let's consider this set of options: SYCL_CTS_ENABLE_EXT_ONEAPI_TESTS=ON SYCL_CTS_ENABLE_EXT_ONEAPI_MEMCPY2D_TESTS=OFF. With the current patch the second option is ignored.

Perhaps the semantics could be that SYCL_CTS_ENABLE_EXT_ONEAPI_TESTS has an effect only on other options that have not explicitly set either ON or OFF.
But the problem with boolean options in CMake is that they are considered as OFF. https://cmake.org/cmake/help/latest/command/option.html
A solution could be to have these options as strings, so now we can have tri-state ON, OFF and nothing.
That said, is this something to pursue?

Perhaps the semantics could be that SYCL_CTS_ENABLE_EXT_ONEAPI_TESTS has an effect only on other options that have not explicitly set either ON or OFF. But the problem with boolean options in CMake is that they are considered as OFF. https://cmake.org/cmake/help/latest/command/option.html A solution could be to have these options as strings, so now we can have tri-state ON, OFF and nothing. That said, is this something to pursue?

Couldn't this be achieved by simply passing SYCL_CTS_ENABLE_EXT_ONEAPI_TESTS as the default value for each of the other options? I.e.,

add_cts_option(SYCL_CTS_ENABLE_EXT_ONEAPI_SUB_GROUP_MASK_TESTS
    "Enable extension oneAPI sub_group_mask tests" ${SYCL_CTS_ENABLE_EXT_ONEAPI_TESTS})

Couldn't this be achieved by simply passing SYCL_CTS_ENABLE_EXT_ONEAPI_TESTS as the default value for each of the other options? I.e.,

add_cts_option(SYCL_CTS_ENABLE_EXT_ONEAPI_SUB_GROUP_MASK_TESTS
    "Enable extension oneAPI sub_group_mask tests" ${SYCL_CTS_ENABLE_EXT_ONEAPI_TESTS})

That was my initial thought too, but the downside to it is that the other options won't change if you set SYCL_CTS_ENABLE_EXT_ONEAPI_TESTS after the initial configuration.

The use-case in mind for this option was to allow for enabling all extension tests in the given group (here oneapi but the same strategy could be used wider) so automated systems won't have to be changed every time a new extension gets tests in the suite. As such a simple overwrite is sufficient, but if it is possible to make it work with selective disabling in tandem with this option, it shouldn't break the use-case.

@MathiasMagnus any feedback on this?

That was my initial thought too, but the downside to it is that the other options won't change if you set SYCL_CTS_ENABLE_EXT_ONEAPI_TESTS after the initial configuration.

Fair, but couldn't the same argument be made in reverse then? I.e., once SYCL_CTS_ENABLE_EXT_ONEAPI_TESTS is set, it is no longer possible to selectively disable a subset of the extension tests?

The use-case in mind for this option was to allow for enabling all extension tests in the given group (here oneapi but the same strategy could be used wider) so automated systems won't have to be changed every time a new extension gets tests in the suite.

Yes that makes sense. In that scenario changing the configuration later on wouldn't be an issue though. The CI setup only has to make sure that SYCL_CTS_ENABLE_EXT_ONEAPI_TESTS is set for the initial configuration. An alternative solution to this might be to ship a set of CMake presets.

needs another week